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Abstract 

This paper considers the problem of channel coding with a given (possibly suboptimal) decoding rule. Finite-length 
upper and lower bounds on the random-coding error probability for a general codeword distribution are given. These 
bounds are applied to three random-coding ensembles: i.i.d., constant-composition, and cost-constrained. Ensemble- 
tight error exponents are presented for each ensemble, and achievable second-order coding rates are given. Connections 
are drawn between the ensembles under both maximum likelihood decoding and mismatched decoding. In particular, 
it is shown that the error exponents and second-order rates of the constant-composition ensemble can be achieved 
using cost-constrained coding with at most two cost functions. Finally, saddlepoint approximations of the random- 
coding bounds are given. These are demonstrated to be more accurate than the approximations obtained from the 
error exponents and second-order coding rates, while having a similar computational complexity. 
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I. Introduction 

It is well known that random coding techniques can be used to prove the achievability part of Shannon's channel 
coding theorem, as well as characterizing the exponential behavior of the best code for a range of rates under 
maximum-likelihood (ML) decoding |[T] Sec. 5]. In practice, however, ML decoding is often ruled out due to channel 
uncertainty and implementation constraints. In this paper, we consider the problem of mismatched decoding |2|-|8|, 
in which the decoder chooses the codeword which maximizes a given (possibly suboptimal) decoding metric. In 
this setting, the following random-coding ensembles have been considered (see Section [II] for details): 

1) the i.i.d. ensemble, in which each symbol of each codeword is generated independently; 

2) the constant-composition ensemble, in which each codeword has the same empirical distribution; 

3) the cost-constrained ensemble, in which each codeword satisfies a given cost constraint chosen by the codebook 
designer. 

Most existing work has focused on the achievable rates resulting from these ensembles. Motivated by the fact that 
these rates are strictly smaller than the mismatched capacity in general, the concept of ensemble tightness has 
been addressed (5)-|[8j in order to show that the rates obtained are the best possible for the given random-coding 
ensemble. The goal of this paper is to present a more comprehensive analysis of the above ensembles, including 
finite-length bounds and approximations, error exponents, and second-order coding rates. 

A. System Setup 

The input and output alphabets are denoted by X and y respectively, and the channel is denoted by W(y\x). 
Except where stated otherwise, we assume that X and y are finite, and thus the channel is a discrete memoryless 
channel (DMC). We consider length-n block coding, in which a codebook C = {x^\ . . . , x^ M '} is known at both 
the encoder and decoder. The encoder takes as input a message m uniformly distributed on the set {1, . . . , M}, 
and transmits the corresponding codeword x^ m \ The decoder receives the vector y at the output of the channel, 
and forms the estimate 

n 

rh= argmax ff g(i- j) ,s), (1) 
]e{i,...,M}f =1 

where n is the length of each codeword and xf' is the z-th entry of x^ (similarly for yj). The function q(x,y) 
is called the decoding metric. We assume that q(x,y) is greater than or equal to zero, with equality only if 
W(j/|a;) = 0. This assumption entails no real loss of generality, since otherwise the resulting random-coding rates 
are zero whenever the corresponding input x is used (3j. In the case of a tie, a random codeword achieving the 
maximum in ([TJ is selected. We define q n (x, y) = n"=i q( x iiUi) an d W n {y\x) = Yli=i ^{Vi\ x i)- 

The mismatched capacity of a given channel and metric is defined to be the supremum of all rates R = — log M 
such that the error probability p e (C) can be made arbitrarily small for sufficiently large n. An error exponent E(R) 
is said to be achievable if there exists a sequence of codebooks C n of length n and rate R such that 

liminf lagp e (C n ) > E(R). (2) 

n— >oo tl 
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We let p e (n, M) denote the average error probability with respect to a given random-coding ensemble which will 
be clear from the context. The random-coding error exponent E r (R) is said to exhibit ensemble tightness if 

lira — logp e (n, e nR ) = E r (R). (3) 
n— >oo n 

B. Previous Work 

The most notable early works on mismatched decoding are by Hui [4| and Csiszi^cer and Ki'^cerner (9), who 
independently derived the achievable rate commonly referred to as the LM rate, given by 



7 LM (Q) = sup E 

s>0,a(-) 



q(X,Y) s e a( - x ^ 
log— — =r 



E[q(X,Y) s e a ( x )\Y] 
where (X, Y) ~ Q x W. This rate can equivalently be expressed as 



(4) 



I LM (Q) = mm I p(X;Y), (5) 

PXY 

where the minimization is over all joint distributions on X x y satisfying 

Px{x) = Q{x) (6) 
P Y {y) = Y J Q{x)W{y\x) (7) 

X 

^P XY (x,y)logq(x >y )>J2Q(x)W(y\x)logq(x,y). (8) 

x,y x,y 

For binary-input DMCs, the LM rate is equal to the mismatched capacity after the optimization over Q |5 1. However, 
this is not true in general for non-binary channels (3), (6), (8). 

The derivation of the LM rate in |4) uses random coding in which the empirical distribution of each codeword 
is constrained to be close to a given distribution Q(x). A similar analysis can be performed for the constant- 
composition ensemble, yielding the same result [9|. In either case, the proof is valid only when the input and 
output alphabets are finite. Building on early work by Fischer [10], Kaplan and Shamai |2| performed an analysis 
which does not rely on the alphabets being finite, and derived the achievable rate known as generalized mutual 
information (GMI), given by 

a r q(X,Y) s 

iog „ r/ ; J., T , ■ (9) 



I^ NU (Q) = supE 

s>0 



E[q(X,Y)s\Y] 



In the discrete memoryless setting, it can be shown that 

I GM1 (Q)= min D(P XY \\Qx? Y ), (10) 

PXY 

where the minimization is over all joint distributions on X x y satisfying 

Py(y)=^Q(a;)^(y|x) (11) 

X 

Y,P XY {x,y)logq(x,y)>Y,Q(x)W(y\x)logq(x,y), (12) 

x,y x,y 

The GMI cannot exceed the LM rate, and the latter can be strictly higher even after the optimization over Q. 
Motivated by this fact, Ganti et al. (7J proved that Q is achievable for general memoryless channels. This was 
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done using cost-constrained codes, in which each codeword is constrained to satisfy A X^=l a ( x i) ~ ^qW-^O] 
for some cost function a(-) which can be optimized. 

It should be noted that the optimization of I LM (Q) and I GMI (Q) over Q is questionable in some applications, 
since it requires the codebook designer to know the channel. For the cost-constrained random-coding ensemble, a 
similar statement is true for the optimization over a(-) in Q. On the other hand, mismatched decoding is relevant 
in settings where the channel is known but implementation constraints prohibit the use of the ML decoder, and 
such optimizations are valid in these settings. Throughout this paper, we present results for a given Q. Most results 
presented for the cost-constrained ensemble will assume a fixed set of parameters, but we will sometimes choose 
a( ) as a function of the channel. In such cases, it is assumed that the codebook designer knows the channel. 

In the terminology of [7], <[5j and ( fT0) i are primal expressions, and Q and |9} are the corresponding dual 
expressions. Indeed, the latter can be derived from the former using Lagrange duality techniques JTT). As well 
as extending readily to general alphabets, the dual expressions have the advantage that an achievable rate can be 
obtained by substituting any values of s and a(-) into |4} and |9|, whereas (|5]l and ( fT0| ) only give valid bounds 
after the minimization is performed. 

Due to the lack of converse results in mismatched decoding, it is of interest to determine whether the achievable 
rates are tight with respect to the ensemble average. For the i.i.d. ensemble with input distribution Q, it is known that 
the random-coding error probability p e (n, e nR ) tends to as n — > oo when R < I GMl (Q), whereas p e (n, e ) — > 1 
as n — > oo when R > I GM1 (Q) J5J, j7j. Similarly, for the constant-composition ensemble with input distribution 
Q, it has been shown that p e (n,e nR ) — > as n — > oo when R < I LM (Q), whereas p e (n,e nR ) — > 1 as n — » oo 
when R > I LM {Q) (6), (§. 

Finally, random-coding error exponents under mismatched decoding are presented in Q, (9|, p"2| . The exponent 
in |9j is valid for any DMC, and is proved using constant-composition random coding. Error exponents for the 
i.i.d. ensemble and cost-constrained ensemble are presented in [2| and |12] respectively, and both results apply in 



the case of general alphabets. The exponents in (9), p2) recover the LM rate, whereas the exponent in [2] only 
recovers the GMI. To our knowledge, the ensemble tightness of the exponent has been shown only in the case of 
constant-composition random coding with a sufficiently small rate fl3) . 

C. Contributions 

Motivated by the fact that most existing work on mismatched decoding has focused on achievable rates, the 
main goal of this paper is to present a more detailed analysis of the random-coding error probability. Our main 
contributions are as follows: 

1) Finite-length upper and lower bounds on the random-coding error probability for a general random-coding 
distribution are given. These bounds are applied to three random-coding ensembles: the i.i.d. ensemble, constant- 
composition ensemble and cost-constrained ensemble. 

2) Ensemble-tight error exponents are given for each ensemble, and connections are drawn between the ensembles. 
For the i.i.d. and constant-composition ensembles, our analysis proves the ensemble tightness of the exponents 
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given in [2 1 and |9| respectively. The exponent for the cost-constrained ensemble appears to be new. 

3) Achievable second-order coding rates are presented for each ensemble. We focus on the cost-constrained 
ensemble, for which we derive the second-order rate using novel techniques. 

4) The benefits of cost-constrained random-coding with multiple cost functions are studied. In particular, while the 
constant-composition ensemble gives the best exponents and second-order rates of the ensembles considered, 
it is shown that the same performance can be achieved using cost-constrained coding with two cost functions. 
The latter does not require the input alphabet to be finite, and thus the results hold in greater generality. 

5) Saddlepoint approximations of certain random-coding bounds are presented for the i.i.d. and constant-composition 
ensembles. These are shown to provide accurate approximations to the corresponding bounds at all rates, even 
at small block lengths. In contrast, the approximations obtained from the error exponents and second-order rates 
are generally accurate only at large block lengths, or only for a narrower range of rates. For the i.i.d. ensemble, 
we prove that the saddlepoint approximation yields the same behavior as the corresponding random-coding 
bound as n —> oo with a fixed rate, to within a multiplicative 1 + o(l) term. 



D. Notation 

The set of all probability distributions on an alphabet A is denoted by V{A), and the set of all empirical 



distributions on a vector in A n (i.e. types 1 14 Sec. 2] |15J) is denoted by V n (A). The type of a vector x is denoted 



by P x ( ). For a given Q 6 V n (A), the type class T n (Q) is defined to be the set of all sequences in A n with type 
Q. 

The probability of an event is denoted by P[-], and the symbol ~ means "distributed as". The marginals of a joint 
distribution PxY{x,y) are denoted by Px( x ) an d Py{v)- We write Px = Px to denote element-wise equality 
between two probability distributions on the same alphabet. Expectation with respect to a joint distribution Pxy(x, y) 
is denoted by Ep[-]. When the associated probability distribution is understood from the context, the expectation is 
written as E[-]. Similarly, mutual information with respect to Pxy is written as Ip(X; Y), or simply I(X; Y) when 
the distribution is understood from the context. Given a distribution Q(x) and conditional distribution W(y\x), we 
write Q x W to denote the joint distribution defined by Q(x)W(y\x). The set of values where a probability mass 
function Q is non-zero is denoted by supp(Q). 

For two sequences f(n) and g(n), we write f(n) = g(n) if lim JWOO ~ log = 0, and similarly for < and 
>. We write f(n) = 0(g(n)) if |/(n)| < c|g(n)| for some c and sufficiently large n. We write f(n) = fl(g(n)) 
if g(n) = 0(/(n)), f(n) = Q(g(n)) if both f(n) = 0(g(n)) and f(n) = Q( 5 (n)) hold, and f(n) = o{g{n)) if 
lim«^oo = 0. 

We denote the tail probability of a zero-mean unit-variance Gaussian variable by Q(-), an d we denote its functional 
inverse by Q~ 1 (-). The complementary error function is denoted by erfc(-), i.e. erfc(z) = 2Q(^/2z). All logarithms 
have base e, and all rates are in units of nats except in the examples, where bits are used. We define [c] + = max{0, c}, 
and denote the indicator function by !{•}. 
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E. Structure of the Paper 



In Section |TTJ we formally define the random-coding ensembles used throughout the paper. In Section III 



we 



give upper and lower bounds on the random-coding error probability. In Section IV we derive ensemble-tight error 
exponents for each ensemble, and draw connections between the ensembles. Second-order coding rates are presented 
in Section [V] and saddlepoint approximations are presented in Section VI Conclusions are drawn in Section VII 



II. Random-Coding Ensembles 

In this section, we formally define the three random-coding ensembles described in Section [I] Throughout 
the paper, we denote the random-coding distribution by Px, and we assume that each codeword is generated 
independently. We pay particular attention to the following. 

1) The i.i.d. ensemble is characterized by 

n 

Px(x) = Y[Q(xi). (13) 

i=i 

In words, each symbol of each codeword is generated independently according to Q. 

2) The constant-composition ensemble is characterized by 

Px(x)= 1 l{ a! eT"(Q n )}, (14) 

where Q n is a type such that max^ \Q n (x) — Q(x)\ < — . That is, each codeword is generated uniformly over 
the type class T n (Q n ), and hence each codeword has the same composition. 

3) The cost-constrained ensemble is characterized by 

1 " 

Px(x) = — Y[Q( Xi )l{xeV n }, (15) 



where 



{1 n S 1 

x : - y~]ai(xi) - 4>i <-,l = l,...,L} 
i=l J 



(16) 



and where p, n is a normalizing constant, 5 is a positive constant (independent of n), and for each I € {1, . . . , L}, 
ai(-) is a cost function and (f>i = Eg[a;(X)]. Roughly speaking, each codeword is generated according to an 
i.i.d. distribution conditioned on the empirical mean of each cost function a; (x) being close to the true mean. 
This generalizes the ensemble studied in JT| Sec. 7.3] |T2) by including multiple costs. 
The cost functions {a/} in ( fT5j ) should not be viewed as being chosen to meet a system constraint (e.g. power 
limitations). Rather, they are introduced in order to improve the performance of the random-coding ensemble. 
Thus, one can view the costs as being pseudo-costs, rather than system costs. However, the latter can handled 



similarly; see Section VII for further discussion. 

The constant S in ( fl"5j ) could, in principle, vary with I and n. However, a fixed value will suffice for our purposes. 
The constant-composition ensemble is a special case of the ensemble in ([15}, since it is obtained by replacing Q 
by Qn ( see (114)), setting L = \X\ — 1, and choosing the cost functions cii = (1,0, ... ,0), = (0, 1,0,... ,0), 
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etc., along with 5 < 1. Of course, the i.i.d. ensemble is also a special case of ( fT3) ), since is it obtained by setting 
L = 0. 

The following proposition shows that the normalizing constant [i n in ( fT3| l decays at most polynomially in n. 
When | A" | is finite, this can easily be shown using the method of types. In particular, choosing the functions given in 



the previous paragraph to recover the constant-composition ensemble, we have [i n > (n+ 1 ) C 1^ I i) j 14 pp. 17]. 
For the sake of generality, we present a proof which applies to more general alphabets, subject to minor technical 
conditions. The proof for the case L = 1 is given in (Tl Ch. 7.3]. 

Proposition 1. Fix an input alphabet X (possibly infinite or continuous), an input distribution Q £E V(X) and the 
cost functions ai(-), • • • ,ai(-). If the second moment of ai(X) is finite under X ~ Q for I = 1, . . . , L, then there 
exists a 8 > such that the normalizing constant in ( |15| l satisfies fi n = £l(nT L / 2 ). 

Proof: Defining the vectors <fr — (01, • ■ ■ , 4>l) T and a(x) = (ai(x), . . . , o,l(x)) t , we have 



4=1 < -^=(iTa(Xi) -nd>) -< 4=1 



(17) 



where -< denotes element-wise inequality, and 1 is the vector of ones. We define E to be the covariance matrix of 
a{X) — 4>. Since each value of <pi and of = E[(ai(X) — 4>i) 2 ] is finite by assumption, we can apply the multivariate 



central limit theorem [16 



Sec. VIII], yielding that the distribution of X)ILl a (^«) ~ n< ^J converges to that 
of multivariate normal random variable with zero mean and covariance matrix E. 

We will show that /i„ = 8(n~ L / 2 ) provided that det(E) > 0. In the case that det(E) = 0, the evaluation 
of /i„ can be reduced to a lower dimension V ', yielding = 8(n~ L I 2 ) and thus proving the general result 
(i n = Q(n~ L / 2 ). The proof of the case det(E) > is most easily understood in the case that the density of 
^= ( Y^i-i a (Xi) — ncftj converges to that of a multivariate normal random variable, namely 

/(z) - ( 27 r)^det(E)V 2 eX P(-r TS] 'S- (18) 
Such convergence is not obtained in general, since convergence in distribution does not imply convergence of the 



density |T6| Sec. XV]. 

Since the density in ( fT8] l approaches n^yr/v j ct (£)i/2 as eac h entry of z tends to zero, and since the event in 



( fT7| i constrains each entry to be -JL -close to its mean, we have 

L 



2rt 



(1 + 0(1)) 



Mn (27r)V2 dct(E)V2 (19) 
thus yielding /i n = Q(n~ L / 2 ) as desired. To handle the general case, we make use of Jl7] Thm. 6.4], which gives 
asymptotic expressions for probabilities of sets of the form ( fTB"] ). In particular, the desired result fi n — Q(n~ L / 2 ) 
follows by choosing 6 to be greater than or equal to the largest span of the ai(X) which are lattice random variables. 
In the case that all of the ai(X) are non-lattice random variables, S can be chosen arbitrarily. ■ 
In accordance with Proposition [T| we henceforth assume that the choice of i5 for the cost-constrained ensemble 
is such that \i n = £l(n~ L / 2 ). 
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III. Random-Coding Error Probability 

In recent years, the problem of characterizing the finite-length performance of coded communication systems has 
regained significant attention. In particular, several upper and lower bounds on the smallest error probability for 
a given block length n and number of messages M were provided by Polyanskiy et al. [18]. In this section, we 
obtain finite-length upper bounds on the error probability in the mismatched setting. Furthermore, we address the 
problem of ensemble tightness in the finite-length regime by obtaining lower bounds to the random-coding error 
probability. 



A. Finite-Length Bounds 

Throughout this section, we make use of the random variables 

(X,Y,X) ~ P x (x)W n (y\x)P x (x). 



(20) 



As stated in 1 18 1, the exact random-coding error probability of the decoder that breaks ties randomly is given by 

M-l 



1 



where 



Pe ( n ,M) = i-j2 ( M . 1 ) . 1 . 

1=0 



p (x,y)=V[q n (X,y)=q n (x,y)] 



Po (X,Yy Pl (X,Y) 



M-l-t 



Pl (x,y)=Y[q n (X,y)<q n (x,y)]. 



(21) 

(22) 
(23) 



The computation of ( [21} is generally infeasible even for small values of n. However, we can weaken pi) to obtain 
computationally tractable bounds. The following theorem extends the RCU bound of fl8) to the mismatched setting, 
and provides a weakened bound with a similar form. 

Theorem 1. (Upper Bounds) For any random-coding distribution Px(x), the random-coding error probability 
satisfies 



p e (n, M) < rcu(n, M) < rcu s (n, M), 



where 



rcu(n,M) = E[min {l, (M — l)¥[q n (X, Y) > q n (X,Y)\X,Y]}] 



A 



mu s (n,M) = E 
and (|24[) holds for any s > 0. 



linjl, (M - 1) 



E[q n (X,Y) s \Y] 
q n (X,YY 



(24) 

(25) 
(26) 



Proof: Similarly to |18|, ( |25| > follows by upper bounding the error probability by that of the decoder which 
decodes ties as errors, and then applying the truncated union bound to 



|J {q n {X^,Y)>q n (X,Y)} 



(27) 
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after conditioning on X and Y. We obtain ( |26| > using Markov's inequality. ■ 
As discussed in Section [IJ it is of interest to find lower bounds on the random-coding error probability in the 
mismatched setting. This is addressed in the following theorem. 

Theorem 2. (Lower Bounds) The random-coding error probability for the mismatched decoder which resolves ties 
randomly satisfies 

1 / L 



p e (n,M) > rcu L (n,M) > rcu' L (n,M) > -fl - -Vcu(n, M ), 

2 \ e / 



(28) 



where 



rcu L (n, M) = 1 — ^E [p[g"(X, Y) < q n (X, Y) \ X, Y] M 1 



-E 



M-l 



(29) 



rcu^ (n,M) = 1 - ^E 



exp ( _ ( M _ l)¥[q n (X, Y) > q n (X,Y)\X, Y] 



-E 
2 



exp ( - (M - l)¥[q n (X, Y) > q n (X, Y) \ X, Y] 



(30) 



Proof: Consider a fixed codebook C with error probability p e (C). Let Bq be the event that one or more 
codewords yield a strictly higher metric than the transmitted one, and let Bg (I > 1) be the event that the transmitted 
codeword yields a metric which is equal highest with £ other codewords. We have 



M-l 



IK 



: (C)=P[5„]+^PN — 
» ] + - E F[Bi] 



(31) 



(32) 



(33) 



where p' e (C) = P[B ] + ^'P^] is the error probability of the mismatched decoder which decodes ties as 



errors. Averaging ( |33| ) over the random-coding distribution, we obtain 



p e (n,M) > 



\J{q n (X^\Y)>q n (X,Y)} 



M 



\J{q n (X^,Y)>q n (X,Y)} 



(34) 



The first inequality in ( |28| i follows by writing each probability in p4| ) as an expectation given X and Y, and using 
the fact that for a sequence of independent events A\, . . . , Ak having equal probability, 



.1=1 



1 



(35) 



= l-(l-P[A 1 ]) fe . (36) 
The second inequality in (|2"8j) follows since 1 — a < e~ a . The final inequality in (|28|> follows by lower bounding 
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rc<(n,M) • ~-\ 2 



>-|l-E 



d30ll as follows: 

exp ( - (M - l)F[q n (X, Y) > q n (X, Y) \ X, Y] 

exp (-min jl,(M- l)V[q n (X,Y) > q n (X,Y) \X,Y]\^j 

n {l, (M - l)P[g"(X, Y) > q n (X, Y) \ X, Y] 
where d39]l follows since e -« > (1 - i) a for a e [0, 1]. 



(37) 
(38) 
(39) 



We refer to rcu(n,M), rcu s (n, M), rcu^(n, M) and icu' h (n, M) as random-coding union (RCU) bounds |18| . 
The bounds tcul and rcuj^ have a similar form; the latter is slightly weaker but more numerically stable. From 
1 )28} , we see that rcu and tcul coincide to within the constant factor |(l — ij, and thus accurately describe the 
random-coding error probability. However, for general channels, decoding metrics and random-coding distributions, 
their computation can be challenging. Generally, some degree of symmetry is needed in order to facilitate the 



computation. Since constant factors do not affect the error exponent, we have rcu = tcul = p e - In Section IV this 



observation will be used to obtain ensemble-tight error exponents for the ensembles introduced in Section [TT] 

The bounds rcu and rcu s generally have a similar computational complexity. However, we will see in Section [VT] 
that rcu s can be approximated accurately using the saddlepoint approximation. In particular, for the i.i.d. ensemble, 
we will obtain an approximation fcu s with low computational complexity such that lim„_ i . 00 rcu^n'e""] = •"■ ^ 01 
rates R below the GMI, subject to minor technical conditions. Furthermore, we will use rcu s to derive achievable 
error exponents in Section IV and achievable second-order rates in Section [V] 

The focus in this paper is on memoryless channels with single-letter decoding metrics and the random-coding 
ensembles given in Section [II] However, the above bounds apply to channels with memory, multi-letter decoding rules 
and general random-coding ensembles with independently generated codewords each having the same distribution. In 
fact, the independence assumption is not needed for the upper bounds, and lower bounds in the pairwise independent 
case can be obtained by replacing ([36]l with de Caen's lower bound p9|; see J20| Theorem 1]. 



B. Numerical Results 

We compare the upper and lower bounds numerically by considering the channel defined by the entries of the 
\X\ x \y\ matrix 

1 - 2(5 S S 

51 l-28x Si (40 1 

5 2 S 2 1 — 2<5 2 

with X = y = {0, 1, 2}. The mismatched decoder chooses the codeword which is closest to y in terms of Hamming 
distance. For example, the decoding metric can be taken to be the entries of the matrix 

1-2(5 S S 

8 1-2S S (41) 
<5 5 1-25 
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for any 5 G (0, |). That is, the channel is asymmetric, but the decoder uses a symmetric decoding metric. 

We first set So = 0.01, <5i = 0.05 and 82 = 0.25 and consider the i.i.d. and constant-composition ensembles with 
Q = (|, |, |); the cost-constrained ensemble is omitted because the multi -letter nature of the ensemble makes it 
difficult to compute the finite-length bounds. Under these parameters we have I GMI (Q) = 0.643, I LM (Q) = 0.728 
and I{X] Y) = 0.763 bits/use. Figure [I] plots rcu(n, M ) and rcu^n, M) with n — 45. We confirm the close match 
between the upper and lower bounds predicted by Theorem [2] It should be noted that this gap closes further when 
the decoding metric yields fewer ties J20| Fig. 1]. The present metric yields a significant number of ties due to the 
fact that q(x, y) can only be one of two different values, namely 8 and 1 — 28, 

We see that the constant-composition ensemble yields a lower error probability than that of the i.i.d. ensemble, 
particularly at low to moderate rates. This shows that the performance gain of the constant-composition ensemble 
can be significant even at small block lengths. 

To facilitate the computation at larger block lengths, we consider the symmetric setup of <5 = Si = 82 = 8 = 0.1, 
and focus on the i.i.d. ensemble with Q = (|, |, |). In this case, the decoder is simply the ML decoder, and the 
mutual information is I(X;Y) = 0.633 bits/use. Figure [2] plots the bounds of Theorems [T] and [2] with n = 250. 
For the bound rcu s , we choose s to maximize the random-coding error exponent at each rate R; see Section IV for 
details. Once again, rcu(n, M) and icu! L (n,M) characterize the random-coding error probability accurately over 
the entire range plotted. The bound rcu s (n,M) is only slightly looser than rcu(n, M), The gap between the two 
can be larger at smaller block lengths, and is characterized by a multiplicative 0[^=) term for sufficiently regular 
channels and metrics; see Section [VI] for further discussion. 
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■ rcu(n, M) 

■ rcu s (n, M) 

■ rcu^ (n,M) 



0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 

Rate (bits/use) 



Figure 2. Upper and lower bounds on the random-coding error probability for the channel defined in (40) with n = 250, <5q = 5\ = #2 = 0.1, 
Q = (h I, s) and ^ decoding. 



IV. Random-Coding Error Exponent 

Error exponents characterize the asymptotic exponential behavior of the error probability in coded communication 
systems, and thus provide more information than capacity results alone. In the matched setting, error exponents were 
studied by Fano pTj Sec. 9] and later by Gallager et al. (T| Sec. 5]. The ensemble tightness of the exponent (cf. 
Q) under ML decoding was studied by Gallager (22) and D'yachkov (23) for the i.i.d. and constant-composition 
ensembles respectively. 

In this section, we present a unified analysis for obtaining the error exponents of the random-coding ensembles 
given in Section [II] in the mismatched setting. We prove the ensemble tightness of each exponent, and provide 
several connections between the ensembles. 

We define the sets 



S 



iid 



S CC (Q) = {P XY € V{X x y) : P x = Q} 



V{x x y) 



S cost ({ ai }) = {P XY eV(Xxy) : E P [at(X)] = fr, l = !,••• ,L] 



(42) 
(43) 
(44) 
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and 



V ld {PxY) = {Pxy € V{X x y) : P Y = Py, Ep[log q(X,Y)} > E P [logg(X,F)]} (45) 
T cc {Pxy) = {Pxy e V{X xy) : Px= Px,Py = Py,Ep[logg(X, Y)} > E P [logq(X, Y)]} (46) 
T cost (P XY ,Wi}) = {Pxy eV(X xy) : E^[o,(Jf)]=0i,i = l > -- - ,i, 

P y =P Y , Ep[log g(X,Y)] >E P pogg(X,F)]}, (47) 
where the notation {a{\ is used to denote dependence on ai(-), • • • , ol(-). 

Theorem 3. 77ze ensemble-tight random-coding error exponents for the ensembles defined in ( |13| >-( fT5) > are respec- 
tively given by 



A 



E^ d (Q,R) = min min L>(P X y||Q xW)+ L»(P X y||Q x Py) — R 



A 



E c r c {Q,R)= min min £>(Pxy \\Q x + 7p(X; Y) - R 

PxYtS^iQ) P XY eT"(P XY ) L 



E c r ° st (Q,R,{ ai }) 



A 



£(Pxy||Qx VK)+ D(P XY \\Q x P Y ) - R 



(48) 
(49) 
(50) 



Proof: See Appendix [A] ■ 
The optimization problems in (|4"8]>-(|50l> are all convex when the input distribution and cost functions are fixed. 
Using the method of Lagrange duality {TT| , each exponent can be written in an alternative form. 



Theorem 4. The error exponents in ( |48| )-( |50| l can be expressed as 

E'r d (Q, R) = max E$ d {Q, p) - pR 

pG[0,l] 

E c r c (Q, R) = max E* C {Q, p) - pR 
pe[o,i] 

E™ st (Q,R,{ ai }) = max E c ° st (Q, p,{ ai }) - pR, 
pe[o,i] 



where 



E^ d (Q,p) = sup - log E 

s>0 



E[ q (x,Yy\Y]y 



q(X,YY 



A 



E c c (Q,p)= sup E 

s>0,o(-) 



#o (Q,P,{ai}) = sup 

s>o,{n},{n} 



-logE 
logE 



X 



E[q(X,Y) s e a W \ Y ]\ P 
q(X,Y) s e a ( x ) J 

E[q(X,Y) s e^'=^ a '^-^ \Y] y 

q(X, Y) s e^=i ri(ai(X)-<j>i) 



and (X,Y,X) ~ Q{x)W(y\x)Q{x). 



(51) 
(52) 
(53) 

(54) 
(55) 
(56) 



Proof: The proofs are similar for each of the three ensembles; see Appendix |B] for the proof of i 
We refer to (|48|)-(|50l) as primal expressions for the exponents, and (|5T^>— (|53]> as dual expressions. The exponents 
in ( |49] > and < f5T~| > were derived in |9| and |2| respectively, though both derivations were different to ours. To our 
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knowledge, the alternative expressions in ( |48| and ( |52] ) have not appeared previously, and the exponent £"™ st is 
new. 

The exponents in |5T|)-(|53) can be derived directly, rather than via Lagrange duality. The derivation of ( |5T) is 
presented in (2|, and the expression in |52) follows by combining the achievable error exponent of J24) with the 
fact that under constant composition codes, the metric q(x, y) is equivalent to the metric q(x, y) s e a ^ x ' for any s > 
and a(-) (5). 

The direct derivation of ( |53] > can be considered as a refinement of that of |12| , where it was shown that an 
achievable error exponent in the case that L = 1 is given by 

£^ ost ' (Q, R, ai) = max ££ ost ' (Q, p, d) - pi? (57) 

pS[0,l] 



££ ost (Q,P,ai) = sup- log E 



'E[ g (x,y) s e ai W |y]V" 



(58) 



s>0 

For completeness, we include the derivation of Ey ost here. Applying min{l, a} < a p (p <E [0, 1]) to rcu s in |26 
we obtain 

1 nt^ /E,^ Q n (x)q n (x,y 

V y) s 

where Q n (x) = n™=i Q( x i)> an( ^ ' s defined in ( |16) . From the definition of each codeword a; € 2?„ satisfies 



S \ P 

(59) 



exp 



(r(a?(ac) - n&)) < e MS (60) 



for any real number r, where a" (a;) = Y^i=i a i( x i)- Weakening (|59]l by applying ( |60| > multiple times, we obtain 

Pe(».Af)<- 1+5 MP y Q n (x)W n (y\x) ^ xeV " V ^ L - »r ^ ^ H ( 61 ) 

< t+ M p Q n (x)W n (y\x) ( Wg ( ^i e r ^ ] (62) 

for any real numbers {77} and {r;}. Expanding each term in the outer summation as product from i = 1 to n, we 
obtain 

Since /i„ decays to zero subexponentially in n (cf. Proposition [1} and e p ^MI r! H r! D' 5 is independent of n, we 
conclude that the factor 

(64) 



does not affect the exponential behavior of ( |63| l. We therefore recover the error exponent in ( |53] l. 

Unlike the proof of Theorem [4] this direct derivation does not prove ensemble tightness. A similar statement 
holds for the above-mentioned direct derivations of (f5T|i-(|52]l. However, these have the advantage of extending to 
non-finite alphabets. Under constant-composition random coding, the input distribution Q must have finite support, 
but the output alphabet may be infinite. For the cost-constrained ensemble, we only require that the second moments 
of the cost functions are finite, in accordance with Proposition [T] 
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A. Connections Between the Error Exponents 

The constraints on Pxy and Pxy m <[5 1 ]> — ([53]> are given by the sets defined in (|42|>-(|47|>. Since the constraint 
E P [ai(X)] = <j)i holds by definition when P x = Q, we have that 5 iid C S cost ({ai}) C S CC (Q) for any given set 
of cost functions {a;(-)} and input distribution Q. A similar observation applies to the constraints on Px, and it 
follows that 

K d (Q,R) < E c r ost (Q,R,{ ai }) < E c r c {Q,R). (65) 

This indicates that the constant-composition ensemble yields the best error exponent of the three ensembles under 
consideration. 

By choosing L = 1 and setting ri = ¥\ = 1 in ^56) , we obtain the inequality 

E™ st '{Q,R, ai ) < E c r ost (Q,R, ai ). (66) 
Furthermore, for any set of cost functions {ai}f" =1 , we have 

E c r ost (Q,RAai}ti) < K ost (Q,RA*i}i=i)- (67) 

That is, additional cost functions can never decrease the error exponent. 
In the case that ai(x) does not depend on x, we obtain 

£ r cost (Q,^ai) = E c r ost '(Q,R, ai ) = 4 id (Q,i?), (68) 

and hence we have in general that 

E { ; d {Q,K) < sup£™ st '(Q,i?,ai). (69) 

MO 

The following theorem gives the connection between KJ: ost and £y c , and shows that the two are equal after the 
optimization over Q and di(-). This result is analogous to a connection between the i.i.d. ensemble and constant- 
composition ensemble under ML decoding [24, Theorem 3.4]. The general steps of the proof are similar to that of 
[24 1, but the details are more involved. 

Theorem 5. The function Eq c (Q,p) can be expressed as 

E C C (Q, p) - max sup £ cost '(Q, p, a x ) - (1 + p)D{Q\\Q). (70) 
Q oi(0 

Consequently, 

maxE^(Q,p) = max sup £g ost ' [Q, p,a x ). (71) 

Q Q «i(0 

Proof: The proof of ( |70| > is given in Appendix |c] We obtain |7T} from (7D) since D(Q\\Q) > with equality 
if and only if Q = Q. ■ 
It follows from ( |65| ) and Theorem [5] that E!^ ost is tight with respect to the ensemble average when the optimal 
values of both Q and ai(-) are used. Hence, although E^ ost is a tighter error exponent than £y ost in general, it 
does not improve on the best achievable error exponent using cost-constrained random coding. Furthermore, both 
exponents can be used to prove the achievability of the LM rate and no better. However, the refined exponent 
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£"™ st is useful in the case that one does not have complete freedom in choosing Q and a(-), or when the exact 
optimization over each is not feasible. For example, if the codebook designer does not know the channel, then the 
objective in ( |58] > cannot be computed in order to perform the optimization. Furthermore, even if the channel and 
metric are known, some input distributions may be ruled out due to system constraints (e.g. power limitations). 
The following theorem gives two further connections between the error exponents in the case of ML decoding. 

Theorem 6. Under the ML metric q(x,y) = W^(y|a;), we have 

sup E c r ost ' (Q, R, a) = E [ ; d {Q, R) (72) 
o(.) 

sup E c r ost (Q,R, a) = E c r c (Q,R). (73) 

<*(•) 



Proof: We obtain §T2) by optimizing the objective in ( |54| i over s, and optimizing the objective in ( |58) over 
s and a(-). It was shown in [1 Ex. 5.6] that the optimal value of s in §54\ is equal to Following the same 
steps, we obtain that that the optimal value of s is in ( |58| > is also equal to j^— , and the optimal cost function a(x) 
does not depend on x. Combining these results, we obtain (j72j>. 

From ( |65] l, it suffices to prove that sup a (.) £y ost > 25™ in order to prove ( |73| >. To this end, we will show that 
sup a( -.) EQ OSt (Q, p, a) > Eq c (Q,p) for all p £ [0,1]. The case p = is trivial, so we assume that p > 0. We set 
i = 1, Fi = 1 and r-y = — and write the expectation inside the logarithm of |56| as 



Introducing the distribution = ^^l^xjy, we can write |74| as 



x q(x,y) s 



The summation in ( |75| l coincides with the expectation inside the logarithm of ( |54] >. Furthermore, we have 

MX) 

ga(x) 



D(Q\\Q) = J2 l0 § ^(l) 1 (76) 

a; 

= logE Q [ e a ^)] - ^ Q(x)o(a:) (77) 



where f76] ) follows from the definition of Q, and (|78| follows from the definition of <j> a . Substituting ( |75| ) and ( |78| ) 
into |56| and noting that a suitable choice of a( ) can yield any distribution Q with the same support as Q, we 
obtain 



su P ^ os \Q,p,a) > maxE^iQ^p) - (1 + p)D(Q\\Q). (79) 

a(-) Q 



Taking account of ( |72| i and Theorem|5] the right-hand side of $79\ is equal to Eq c (Q, p), and the proof is complete. 
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Under ML decoding, the exponent E" d is simply Gallager's random-coding error exponent fTl Sec. 5.6], while 



E£ c is Csiszi^cer's random-coding error exponent 1 14 Sec. 10] (see also (24) for an equivalent dual expression). 
The two exponents are equal after the optimization over Q, but Csiszi^cer's exponent can be higher for a given 
Q 1 15|. Theorem [6] shows that using an optimized cost function a( ), we can achieve Csiszi'^cer's exponent in the 



matched setting using one suitably chosen cost function, rather than constant-composition codes. The following 
theorem shows that a similar result holds in the mismatched setting using two suitably chosen cost functions. 

Theorem 7. For any decoding metric q(x,y), we have 

sup E c r ost (Q,R,{a 1 ,a 2 }) = E c r c (Q,R). (80) 

eti(-),a 2 (-) 

Proof: Setting L = 2 and choosing r± = r± = ¥2 = 1 and r = — , the expectation in ( |56} becomes 

l.Q(*)w(y\x) [ g(a>l/) . e a lW+fe ) < 81 > 

Defining Q(x) = j|~j~lirx7j an d following identical steps to |75}-([79}, we obtain 

sup E c ost (Q,p,{a 1 ,a 2 })>maxsupE c ost '(Q,p,a 1 )-(l + p)D(Q\\Q). (82) 

ai(-).°2(-) Q ai(') 

By Theorem |5] the right-hand side of ( |8"2"} is equal to E^ c (Q,p), and we have thus recovered the exponent Ef c . 
The matching lower bound on E^ ost is given in ( |65} . ■ 

We conclude this subsection by showing that ([80} can be derived directly by a suitable choice of the cost functions 
ai(-) and aa(-)- An advantage of this approach is that it provides an explicit formula for o^O) in terms of s and 
cii(-). Furthermore, from the proof of Theorem |6j we can set s = and ai(x) = in the case of ML decoding 
without loss of optimality, and hence all of the optimization parameters are known analytically in this case. 

Fix the input distribution Q, and the parameters p, s > and a(x). We set a\(x) = a(x) and 

\s p a{x) \ P 



a 2 (x) = log W(y\x) ^ g(g|g) , eaM 



(83) 



Using ( |6"T} , we have 



where Q n (x) = Y\2=i Q( x i) anc ^ o"(sc) = 2~Z™=i a ( x i)- We write the summation in ( [85} as 

The constraint on a 2 (x) in the definition of T> n in ( [16} implies that, for all x € T> n , the logarithm in ([86} differs 
from its mean by no more than 5. Expanding each term in the logarithm as a product from 1 to n, we see that this 
mean is simply uEq c (see ([55}), and thus ( [86} is upper bounded by 

ex.p(nE^(Q,p)+d). (87) 
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The derivation is concluded by substituting ( |87] > into ( |85| ), using Proposition [T] and maximizing over p 6 [0, 1], 
s > and a(-). 

This analysis in ([83j)-(|87]i extends immediately to general alphabets, provided that the second moments of a\(X) 
and a,2(X) are finite under X ~ Q (see Proposition [TJ. In the discrete memoryless setting, we relied on the primal 
expressions to show that we can do no better than E!f c . To see that this is true more generally, we lower bound 
as follows: 

/ E[q(X,Y) s e^=i ,i, i(ai(x)-'t>i) |y 



E c c (Q,p)>E 



E 



> 



- logE 
-logE 
logE 



q{X, Y) s e^i J =i r i( a d x )-<l>i) 
E[q(X,Y) s e^'=iri( a i(X)-4>i) \ Y ] 

E[q(X,Y) s e^'=^ a '^-^ \Y^ P 



X 



X 



(88) 
(89) 
(90) 



q(X, y) s e^i=i n{ai{x)-<t>i) 

where ([88) follows for any s > 0, {a/}, and {f;} by lower bounding the supremum over a(x) by the particular choice 



J2i=i ri(ai(Xi) — 4>i) with 4>i = Eg[ai(X)], (|89j) follows for any {r;} by expanding the logarithm of products as a 
sum of logarithms and noting that the term e^' =1 r i( a i( x )-<Pi) has no effect on the resulting expression regardless 
of what values {n} take, and |90) follows from Jensen's inequality. By optimizing (j90j» over s, {a;}, {r;} and 
{r/}, we obtain the desired result. In summary, we have shown that L — 2 cost functions suffice to achieve Ef, 
and we can do no better by using more cost functions. 



B. Numerical Results 

In this section, we plot the exponents for the channel defined in ( |40"| i under both the minimum Hamming 
distance and ML decoding metrics, again using the parameters 5q = 0.01, <5i = 0.05 and S2 — 0.25. We set 
Q = (0.1,0.3,0.6), which we have intentionally chosen suboptimally to highlight the differences between the error 
exponents when the input distribution is fixed. Under these parameters we have I GMl (Q) — 0.387, / LM (Q) = 0.449 
and I(X; Y) = 0.471 bits/use. 

We evaluate the exponents using the optimization software YALMIP J25) . We allow each of the optimization 
parameters (e.g. s and a(-)) to vary with R. The exponent Ef ost is optimized over ax(-), performing the optimization 
jointly with s in d58j. The cost function for £y ost (L = 1) is chosen using an alternating optimization between ai(-) 
and (s,r, r) in \56) , using the ai(-) which maximizes i?Q OSt as a starting point and terminating when the change 
in the exponent between iterations becomes negligible. To demonstrate the effectiveness of this procedure, the cost 
functions for Ef ost (L = 2) are chosen using an alternating optimization between (ai,a 2 ) and (s,r 1 ,r2,ri,r : 2), 
initially setting both ai(-) and a 2 (-) to be equal to the ai(-) which maximizes EQ OSt . From Theorem |7| we expect 
this to recover the exponent E£ c . 

From Figure[3] we see that E^ c and £y ost (L ~ 2) are indistinguishable, indicating that the alternating optimization 
technique was effective in finding the true exponent. The exponent E^° st (L — 1) is only marginally lower, while 
the gap to £y ost is larger. The exponent E^ d is not only lower than each of the other exponents, but also yields 
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Figure 3. Error exponents for the channel defined in \40) with <5o = 0.01, <5i = 0.05, &2 = 0.25 and Q = (0.1,0.3,0.6). The mismatched 
decoder uses the minimum Hamming distance metric given in j41) . The corresponding achievable rates I GMl (Q), 7 LM (Q) and I(X;Y) are 
respectively marked on the horizontal axis. 



a worse achievable rate. This example demonstrates that for a fixed Q, the refined exponent E^ ost (L — 1) can 
outperform E^ ost even when ai(-) is optimized. 

V. Second-Order Coding Rates 

In the matched setting, the finite-length performance limits of a channel are characterized by M*(n, e), defined 
to be the maximum number of codewords of length n yielding an error probability not exceeding e for some 
encoder-decoder pair. For channels satisfying the strong converse, M*(n, e) satisfies 

log M*(n, e) = nC + o(n) (91) 

for all e £ (0, 1), where C is the channel capacity. The problem of finding higher order terms in this expansion was 
studied by Strassen [26], and later revisited by Polyanskiy et al. fl8) and Hayashi (27), among others. For DMCs, 
the second-order expansion is of the form 

log M*(n,e) = nC - VnV Q- 1 (e) + 0(logn), (92) 

where V is a constant known as the channel dispersion. The expansion in (J92j» is often referred to as the normal 
approximation, and is said to describe the second-order coding rate. 

In this section, we present achievable second-order coding rates for the ensembles given in Section [I] i.e. 
expansions of the form |92| with the equality replaced by >. To distinguish between the ensembles, we define 
M lld (Q, n, e), M cc (Q,n,e) and M cost (Q, n, e) to be the maximum number of codewords of length n such that 
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the random-coding error probability does not exceed e for the i.i.d. ensemble, constant-composition ensemble and 
cost-constrained ensemble respectively, using the input distribution Q. For the cost-constrained ensemble, we choose 
L = 2 and allow the cost functions to be optimized. More specifically, we will fix one cost function arbitrarily and 
choose the other one in terms of W, q, Q, and the first cost function. 

A key quantity in the second-order analysis for ML decoding is the information density, given by ]28) 



where Q is a given input distribution. In the mismatched setting, two generalizations of i(x,y) are given by 

i a (x,y)^ log ^ y ]"_ (94) 

« s ,a [x,y)= log , . , (95) 



where s > and a(-) are parameters. Similarly to Section IV we will see that o(-) represents a mathematical opti- 
mization parameter for the constant-composition ensemble, whereas for the cost-constrained ensemble it represents 
one of the cost functions. The information density is recovered from (|94l>-(|95| by setting q(x, y) = W(y\x), s = 1, 
and a (a;) = 0. We write 

n 

A 



i"(ac,y) = ^2i s (xi,yi) (96) 

n 
i=l 

as well as the usual notations Q n (x) = Y[?=i Q( x i) an d a n (x) = X)"=i a ( x i)- We define the quantities 

I s (Q)=E[i s (X,Y)} (98) 
U S (Q) = Var[ ls (X,F)] (99) 
I Sia (Q)=E[i Sta (X,Y)} (100) 
V s . a {Q) = E[Var[z s , Q (X, Y) \ X}] , (101) 

where (X, Y) ~ Q x W. From Q and d5J, we see that the GMI (respectively, LM rate) is equal to I S (Q) 
(respectively, I s , a {Q)) after optimizing over s (respectively, s and a(-)). 

We state the second-order coding rates for the i.i.d. and constant-composition ensembles without proof, since 
their analysis is similar to that of the matched setting fTF) , p5) , J29) . For the i.i.d. ensemble we obtain 

logAf iid (Q,n,e) > nI s (Q) - y/nU,(Q) Q- X (e) + 0(1) (102) 

for any Q and s > 0. Similarly, for the constant-composition ensemble, we obtain 



log M cc (Q,n,e) > nI., a (Q) ~ sJnV^M Q~ l {e) + O(logn) (103) 

for any Q, s > 0, and a( ). These expressions can also be obtained by following the proof of Theorem [8] below. 
The corresponding results for the matched setting are obtained by setting s = 1 and a(x) = 0. 
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The constant-composition ensemble yields two advantages over that of the i.i.d. ensemble: the additional parameter 
a(-), and a conditional (rather than unconditional) variance in the second-order term. In the matched setting, these 
advantages vanish, since the choice a(x) = is optimal, and the conditional and unconditional variances coincide 
under the capacity-achieving input distribution |18|. Nevertheless, when system cost constraints are present, the 
conditional variance can be strictly higher than the unconditional variance after optimizing Q, even under ML 
decoding. 

The main result of this subsection is the following theorem, which states that the cost-constrained ensemble yields 
the same second-order expansion as ( |103| l. We first present a proof for DMCs which yields a o(y/n) third-order 
term, and then discuss the changes required to obtain a O(logn) third-order term and handle general alphabets. Our 
proof differs from the usual proof using threshold-based random-coding bounds JT8j, p6| , but the latter approach 
can also be used in the present setting |30|. Our analysis can be interpreted as performing a normal approximation 
of the bound rcu, in (T26b. 



(104) 



Theorem 8. Using cost-constrained random-coding with L = 2, 

log M cost (Q,n,e) > nI s>a {Q) - ^nV s , a {Q) Qr 1 {e) + o{^) 
for any s > and a(-). 

Proof: Throughout the proof, we make use of the random variables (X,Y,X) ~ Q(x)W (y\x)Q(x) and 
(X, Y, X) ~ Px(x)W n (y\x)Px(x). Probabilities and expectations containing a realization x of X are implicitly 
defined to be conditioned on the event X = x, and similarly for realizations x of X. 
We choose the cost functions 



We weaken rcu s in ( |26] l as follows: 

rcu s (n, M) = E 

< E 

< E 



a\{x) = a(x) 
a 2 {x) = E[i s>a (x,Y)]. 

\ ' V ' q n {X 1 YY 

q n (X, Y) s e an ( x ) 
Me 26 ^Q n (S)g™(X,Y) s e a "W 
' ~^Ln q n (X, Y) s e an ( x 1 

Me 25 " 1 

l ; i !a (X,r)+logC/<log — 



(105) 
(106) 

(107) 
(108) 
(109) 
(HO) 



where (jTOHJ follows from the constraints on a n (x) in ( |16| l, ( |109| l follows by substituting the random-coding 
distribution in ( fl3| l and summing over all x instead of just x g T> n , and \\ \Q\ follows from the definition of 
i St a and the identity E[min{l, A}} = P[A > U], where U is uniform on (0, 1) and independent of A. 
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For any set A n C T> n , implies 



p e (n, M) < F[X <£ A n ] + max 



*«„(*, Y) + log *7< log 



Me 



2(51 



Choosing M such that 



log M = nI s , a (Q) - log fin -25-/3 

for some /?, we obtain 

P e (n, M) < ¥[X i A n \ + max pfc B (aj, Y) + log £7 < n/ s , a (Q) - 

xeA„ 

Choosing 



(111) 



(112) 



(113) 



(114) 



6' 



6' 



(115) 

(116) 
(117) 



A n = |x e 2?„ : Var[»7 )B (sB,y)] < n(E[Var[i a ,„(X, Y) | *]] +5') 
for some 5' > 0, we obtain 

¥[X £A n ] = p x{x)U\&v\il a {x,Y)] > n(E[Var[i s , a (X,Y) | X] 

< — Z!Q n (^)l(varK ia (x,Y)] > n(E[var[ ls , Q (X, Y) | X] 

< e -n(J5(5')+o(l)) 

where ( |117| i holds for some £?(<$') > by Hoeffding's inequality for bounded random variables J3~T) , and since 
/i„ decays at most polynomially in n. Furthermore, from the definitions of A n and T> n and the choice of the cost 
function in ( |106| >, we have for any x E A n that 

E\il a (x,Y)]>nE[i s>a (X,Y)]-6 (118) 

Var[tJ |0 (aj,y)]<n(E[var[i., a (A:,Y)|Jf]] +5'), (119) 

where S is the parameter to the ensemble appearing in ( [To} . Since the moments of log U are finite, this implies 

E\il a (x,Y)+logU}>nE[i s , a (X,Y)]+0(l) (120) 

VarK i a (a;,Y) + log[/] < u(e [Var [i., a (X, Y) \ X]\ + J') +0(1). (121) 

Thus, applying \\ \1\ and the Berry-Esseen theorem for independent and non-identically distributed random variables 



T6) Sec - XVI.5] to pT3] |, we obtain 

p e (n,M)<Q[ 



+ 0(1) 



O 



(122) 



'n{V s , a {Q)+5') +0(1) J 

where we have used the fact that the third absolute moment of i sa (X, Y) is always bounded when the alphabets 
X and y are finite [18 Lemma 46]. By straightforward rearrangements and a first-order Taylor expansion of the 
square root function and the Q _1 function, we obtain 

P < sfn(V s>a (Q) + S>) Q" 1 ^) + 0(1) (123) 

(124) 



V^,«(Q)Q _1 (p e ) + 0(Vn*')- 
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The proof is concluded by combining ( |1 12\ and ( |124[ ), taking 5' — > 0, and noting from Proposition [T] that log/x„ = 
O(logn). ■ 
The third-order term o(y/n) in ( |104[ ) can be improved to O(logn) by introducing a third cost function 03(2) = 



Var[i s a (x, Y)}, thus constraining Var[«™ Q (a;, Y")] to be close to its mean, similarly to E[i™ a (a;, Y)] in ( |1 18| >. We 
believe that, after optimizing over s and a( ), the expansions in ( |102| )-( fT04"] i hold with equality, and thus the analysis 
is tight with respect to the ensemble average. However, thus far we do not have a proof which holds for an arbitrary 
metric q{x,y). 

The expansion in ( |104| i applies to channels with general alphabets, under some technical assumptions. In order to 
apply Proposition [T] we require each of the cost functions to have a finite second moment. Furthermore, Hoeffding's 

A 



inequality in (|TTV]» cannot be applied to unbounded random variables. Defining v a ^ a {x) = Va,r[i s . a (x, Y)] and 
X' ~ Q n , it must be checked that the quantity 



1 n 

»\-J2 v Ux' t )~v s . a (Q)>$' 

In * — ' 

i=l 



(125) 



decays to zero sufficiently quickly (not necessarily exponentially fast) for all 6' > 0. Similarly, when i s , a (x, y) is not 
bounded, we must include a constraint on the third absolute moment t SM (x) = E[\i SM (x, Y) — I s .a(Q)\ 3 ] m the set 
A n , and ensure that the analogous quantity to ( |125| l decays to zero sufficiently fast for some 5' > 0. Alternatively, 
one can take the approach outlined in the previous paragraph and let v s , a (-) and t s , a (') be cost functions in the 
ensemble. These will have a finite second moment provided that the sixth moment of i Sja (X, Y) is finite, and in 
this case no further assumptions are required. 

VI. Saddlepoint Approximations 



A limitation of the random-coding bounds given in Section III is that they are often too complex to compute. In 
this section, we show that saddlepoint approximations |32| can be used to accurately characterize certain random- 
coding bounds with a computational complexity comparable to that of the error exponents and second-order rates. 

A. Motivation 

Chernoff-type bounds provide an estimate of the tail probability of a random variable via the cumulant transform, 
namely ¥[( > z] w e K ' T ^ TZ , where k(t) — logE[e T< =] and f = argmin T k(t) — tz. Saddlepoint approximations 
provide more accurate estimates of the form P[£ > z] « a(n, f) • e K ( T )- rz . In its classical form, the coefficient 
a(n,f) is given by [32, Sec. 2.2] 

a(«,f)= — , 1 (126) 

f A /27TK"(f) 

This expression has been used previously in the information theory literature, e.g. see {T Appendix 5A]. It often 
yields accurate approximations, but it is inaccurate when f is close to zero, which occurs when z is close to the 
mean E[(}. In such cases, and more generally, a better coefficient is given by 



2 V V 2 



a( K ,r) = hvfcx(f J ^P-), (127) 
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where erfcx(x) = eric(x) cxp(x 2 ). An asymptotic expansion of erk(x) as x — > oo recovers ( |126| >. However, as z 
approaches E[£] from above, we have f — > ^j»7gy , where k"(0) is the variance of £. In this case, we have 

/ Ik"(t)\ . 

(128) 



tC >^«ierfcx(f^S)e^)- 



«Q(^Y (129) 
We thus recover a normal approximation to the probability for values of z close to the mean E[(}. 

B. Approximation for the i.i.d. Ensemble 

In this subsection, we consider the approximation of the weakened RCU bound rcu s (n,M) under the i.i.d. 
ensemble. We begin by presenting a heuristic derivation of the saddlepoint approximation. A rigorous derivation 
including error bounds is given in the proof of Theorem [9] below. 

Using the identity E[min{l, A}] = V[A > U], where U is uniform on (0,1) and independent of A, we can 



express rcu s (n, M) as 



where 



rcu s (n,M) =P[C„ > 0], (130) 



C n =log^^^(X,ia (13D 

and i"(X, Y) is defined in |96|. This form of rcu s is more amenable to the saddlepoint approximation, since it is 
written as the tail probability of a sum of n + 1 independent variables, n of which are identically distributed. 

Roughly speaking, we will approximate rcu s by expressing ( |130| l in terms of the cumulant transform k„(t) = 
logE[e T< '"], and then applying a suitable approximation to k„(t). Since the cumulant transform is the logarithm 
of the Laplace transform of the probability density function (z) of ( n , the density function itself is expressible 



as an inverse Laplace transform |32 Sec. 1.2], namely 

1 rT+jca 

fu(z) = lT-- e^- TZ dr (132) 

for any f > such that the integral converges. From ( |130| >, rcu s (n,M) is obtained by integrating ( |132[ ) over 
z E [0, oo). Interchanging the integration order, we obtain 





pf+jOQ 




1 T—jao 








/ f—joo 



rcu s (n,M) = — / J e K " (T) - TZ dzdT (133) 

e K " (r) -dr. (134) 

T 



Next, we compute the cumulant transform as 



Kn(r) = logE 



rlog(M - 1) - log(l - r) + logE 



E[q n (X,Y) s \Y] 



(135) 
(136) 



q n {X,YY 

rlog(M - 1) - log(l - r) - nE^{Q, r, s), (137) 
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where ( |137| > follows by expanding the expectation in ( |136[ ) as a product from 1 to n and defining E^ d {Q, r, s) as 
in ( |54| i, with a fixed value of s > rather than a supremum. The second term in ( jl37| i is due to the expectation 
over U, and yields the condition t < 1 for the convergence of the cumulant transform. Substituting ( |136| l into ( |134[ ) 
and replacing M by Af + 1, we obtain 

i rp+joo i 

rcu 8 (n,M + l) = — / e «(p«-^o («,/>,.)) dp (138) 

27r J ip-joo PI 1 - P) 

where we have renamed r as p. 

We choose p to minimize the exponential term in the integrand in (|138|l, namely 



p = arg max £ft d (Q, p, s) - pR, (139) 

P6[0,l] 

where the restriction p E [0, 1] arises from the above observation that k 71 (t) diverges when r is outside this range. 
While k„(t) also diverges when r = or r = 1, we can allow for these values by treating them as limiting cases. 
The dependence of p on Q, R and s is kept implicit. As discussed by Gallager, JT], Sec. 5.6], p is a decreasing 
function of R which equals one for all rates below a critical rate, and approaches zero as R approaches the highest 
achievable rate, which is equal to I S (Q) in (j98j in the present setting. 

The key step in the derivation is to approximate the exponential term in ( |138| l by performing a Taylor expansion 
about p = p in the exponent. Neglecting terms of order higher than two, we obtain 

pR - £ id (Q, p, s)npR- 4 id (Q, p, s) + ^(p - p) + l -c 2 {p - p) 2 , (140) 



where 

Ci ^ R _SE^Q, P ,s) 



(>2 



Sp 

_ S 2 E^ d (Q,p,s) 



(142) 



5p 2 

Applying the approximation ( |140| i to ( |138| l, we obtain a quantity which can be written in terms of the error function 
and i?o ld , and can thus be evaluated numerically with low complexity. Specifically, we use p ^i_ p ^ = ~ p + T3^> a PPl y 

Sec. 2.1]: 



the change of variables p = — -A, and evaluate the following integrals similarly to |32 



, l-,-/^2 erfC H (1 ^VT'- Cl fe'' 
'x 2 — 2xy). 1 

fcuf (n, M) = a iid (p, s ) e -n(Er(Q,p,s)-pR)^ (145) 



where erfcxi(ir, y) = erfc(x — y) exp(x 2 — 2xy). We thus obtain the saddlepoint approximation 



where 



iid^- _\ A l„r__. f - / nc 2 . / n \ , l_„ r .__ ^ jnc 2 



a" d (p, S ) = -erfc Xl (pJ^, Cl J— I +-erfcx 1 ((l-p)^,- Cl W— ). (146) 
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The following theorem shows that rcu" is asymptotically equivalent to rcu" as n — > oo for any rate R which 
yields a positive error exponent. 



Theorem 9. Fix Q, s > and R G (0, I S (Q)), where I S (Q) is defined in ((98). Let R cr be the highest rate such 
that p — 1 for all R < R cl -. For the i.i.d. ensemble, we have the following: (i) If < R < R CT , then 

lim rCU % (n ' 6 J = 1 (147) 
n->oo rcu s (n, e nli ) 

and 

a iid (p,s) = l + o(l). (148) 
(iij If R C T < R < I S (Q), then ( |147[ ) holds provided that i s (X,Y) is a non-lattice variable. Furthermore, we have 

^' ^ (1 - p)p\/2irnc 2 ^ 
regardless of whether i s (X, Y) is a lattice variable. 

Proof: See Appendix [D] ■ 
The right hand sides of ( |148| l and ( |149| l yield precise estimates of rcu' s ld as n — > oo, but the coefficient in ( |146[ ) 
is more accurate at finite values of n, particularly when p is close to zero or one (i.e. values of R just below I S (Q) 



or just above J? cr ). This observation is analogous to the comparison between ( |126| l and ([127) in Section VI-A| 

The proof of the second part of Theorem [9] relies on i s (X,Y) being a non-lattice variable. This is true for 
most decoding metrics and choices of Q and s in the non-binary and/or non-symmetric case. Having computed 
the approximation for several cases inducing a lattice variable, we have observed no noticeable difference in 
accuracy compared to the non-lattice case. In particular, the following example induces a lattice variable, and the 
approximations are still remarkably accurate. 

We consider the example given in (|40)-(|4T), using the parameters So = 0.01, 5i = 0.05, S 2 = 0.25, and 
Q = (|, |, |). Under these parameters, we have 7 GMI (Q) = 0.643 and R CI = 0.185 bits/use. For the quantities for 
rcu s and fcu s , we choose the free parameter s to be the value which maximizes the error exponent at each rate. 
While this is not necessarily optimal at finite values of n, we will see that it yields good random-coding bounds. 
In Figure [4] we plot the error probability as a function of the rate with n = 45. We observe that rcu s and fcu s are 
nearly indistinguishable at all rates. Theorem [9] indicates that this should be the case for sufficiently large n, but it 
is interesting to observe that the same holds true even at small block lengths. 

The curve e~ nE <- approximates rcu s accurately at low rates, but it is pessimistic at high rates. The normal 
approximation is obtained by rearranging ([102} with the O(l) term omitted, using the value of s which achieves 
the GMI in |9). While its behavior is somewhat similar to that of rcu s , but it is less precise than the saddlepoint 
approximation, and it is pessimistic at low rates. It is worth noting that the the computational complexity of the 
saddlepoint approximation is similar to that of both the normal approximation and e~ nEr . 

Our focus thus far has been on approximating rcu s (n, M). As shown in [33 1, one can apply similar techniques 
to approximate threshold-based random-coding bounds. The direct approximation of rcu(n, M) appears to be more 
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Figure 4. Random-coding bounds and saddlepoint approximation for the channel and metric defined in j40j-|4TJ using the i.i.d. ensemble. 

ill) 
3' 3' 3 J 



The parameters are n = 45, S = 0.01, Si = 0.05, <5 2 = 0.25 and Q = (U, i) 



difficult, due to the conditional probability ¥\q(X, y) > q(x, yu . While rcu coincides with rcu s for certain channels 
and metrics (e.g. binary-erasure channel (BEC) with ML decoding), the former is tighter in general. In particular, 
for sufficiently regular channels, the RCU bound with ML decoding achieves a pre-factor which is 0(n _ 5( 1 +' 3 )) 
for rates above the critical rate j 34 Thm. 1], Theorem [9] gives a 0(n~^) pre-factor, which coincides with 1 34 
Thm. 1] only for irregular channels resembling the BEC, and not for regular channels yielding the Q(n^i ( - 1+ ^) 
pre-factor. Based on this observation, we have plotted the quantity n~%ica s in Figure]^ Surprisingly, this quantity 
closely approximates rcu; the deviation is most noticeable at high rates. 

To facilitate the computation of rcu and rcu s at larger block lengths, we consider the symmetric setup of 
So = Si = 82 = 5 = 0.1. Under these parameters, we have I(X;Y) — 0.633 and i? cr = 0.192 bits/use. In Figure 
[5] we plot the rate required for each random-coding bound and approximation to achieve a given error probability 
e = 10~ 8 , as a function of n. Once again the curves corresponding to rcu s and fcu s are nearly identical, and 
similarly for rcu and n~2fcu s . The bound e~ nE " d yields similar behavior to rcu s at small block lengths, but the 
gap widens at larger block lengths. 

In contrast to similar plots with larger choices of e (e.g. fT8} Fig. 8]), the normal approximation is inaccurate, 
particularly at small block lengths. In the present example, the normal approximation is pessimistic, in the sense 
that it underestimates the achievable rate for a given (n, e) at all values of n plotted. However, in other cases 
the approximation can be optimistic. For example, changing the crossover probability to 6 = 0.01, the analogous 
plot to Figure |5] gives a normal approximation curve above that of rcu(n, M) at all block lengths shown. Further 
reducing the crossover probability to S — 0.001, the gap between rcu and the normal approximation becomes even 
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Figure 5. Rate required to achieve a target error probability t for the channel and metric defined in {40)-{4TJ using the i.i.d. ensemble. The 
parameters are e = 10~ 8 , Sq = S\ = 82 = S = 0.1 and Q = (A, A, |). 



larger than that of Figure [5] but in the opposite direction. In contrast, the error exponent approximation for the i.i.d. 
ensemble is never optimistic in this sense, since it can be obtained by weakening rcu s . 

C. Discussion 

1) Relation to the Normal Approximation: In this subsection, we show that the normal approximation can be 
obtained by applying a limiting argument to the saddlepoint approximation as p — > 0. Recall the definitions of 
I S (Q) and U S (Q) in (|98|-(|99|. Consider a rate R which approaches I S (Q), so that p in ( |139[ ) tends to zero. We 
consider the Taylor expansion in ( |140| i with zero in place of p, yielding the approximation 

pR - 4 id (Q, p, s)n(R- I s (Q))p+ \u s {Q) P 2 . (150) 

From ( |139| l, we expect p to be approximately equal to be value obtained be letting the derivative of the right-hand 
side of ( |150[ ) equal zero, i.e. p « ^/^gy^ ■ Within the same order of accuracy, we approximate C2 in ( |142| i by the 
second derivative of ( |150| l, i.e. c-2 U S (Q). The constant c\ does not need to be approximated, since it is precisely 
zero whenever p ^ 1. Substituting these parameters into ( |145| l, we obtain 

fcuf (n,M) « ierfc((/ s (Q) - *)){^)- (151) 

Fixing fcu" d (n,M) to a target value e, applying erfc(z) = 2Q(\/2z), and performing some simple algebra, we 
obtain an expression of the form ( |102[ ), with the 0(\ogn) term omitted. 
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2) Approximation for the Constant-Composition Ensemble: Here we give a heuristic derivation of a saddlepoint 
approximation of rcu s for the constant-composition ensemble. We do not prove a result analogous to Theorem [9] 
but we give a numerical example for which the approximation is accurate. 

We introduce an arbitrary function a(x) and write a n (x) = Y17=i a ( x i)- We approximate rcu s as follows: 

q n (X,Y) s e a "( x ) 

2ZnQ n (x)q n (x,Y) s e a "^ 
q n {X : Y) s e a "W 



rcu s (n, M) = E 
rj E 
= E 



min|l, (M — 1) 
min 1 1 , M 
mm{l,Me^ x > Y A 



(152) 
(153) 
(154) 
(155) 
(156) 



In ( |152[ ) we have used the fact that a n (x) = a n (x) for any function a(-) and codewords x, x of the same type. In 
( |153[ ), we approximate the inner (but not outer) codeword distribution Px(x) by the i.i.d. distribution Q n (x), and 



we replace M — 1 by M. The third and fourth steps are identical to those for the i.i.d. ensemble in Section VI-B 
and ( |156| l holds for an arbitrary x 6 T n (Q) by symmetry, where Y ~ T / l / ™(-|a;). 

The substitution of the i.i.d. ensemble in ( |153[ ) is a heuristic step, and the approximation is pessimistic in the case 
that the cost function a( ) is chosen suboptimally. However, based on numerical results, the approximation appears 
to be accurate when a(-) is chosen to maximize Eq c (Q, p, s, a) for the given value of s, where Eq c (Q, p, s, a) 
denotes the function Eq c in ( |55| i with fixed values of s > and a(-) rather than a supremum. By analyzing ( |156[ ) 
in an identical fashion to the Section VI-B we obtain precisely the approximation defined by ( |145| >-( fl4"6| ), ( |139[ ) 
and ( |141| i-( fT4"2"| ), with Eq c replacing Eq 6 . We denote this approximation by mi c s c a (n, M). 

Figure [6] plots the bounds and approximations for the constant-composition ensemble, again using the parameters 
n = 45, Sq = 0.01, 5i — 0.05, 62 = 0.25, and Q = (|, |, |). For the quantities rcu s and fcu s a , we choose s 
and a( ) to maximize the error exponent at each rate. The normal approximation is obtained by rearranging ( |103[ ) 
with the O(logn) term omitted, using the values of s and a(-) which achieve the LM rate in Under these 
parameters, we have I LM (Q) — 0.728 and R CI — 0.269 bits/use. Despite the fact that its derivation was based on 
heuristic arguments, we see that fcu S! a again approximates rcu s remarkably well, and similarly for ri~%fcu s i0 and 
rcu. 



3) Block Length as a Function of (R,e): The random-coding bounds in Section III are expressed as error 
probabilities in terms of the block length and number of messages. In order to find the required block length n for 
a given rate R and target error probability e, one can perform a bisection search (T8). An advantage of the error 
exponent approximation and normal approximation is that they can be inverted to find an explicit formula for n in 



terms of (R, e). The error exponent approximation p e 



-nE" d 



yields the approximation 



loge 



E? A (Q,Ry 



(157) 
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Figure 6. Random-coding bounds and saddlepoint approximation for the channel and metric defined in |40}-j4TJ using the constant-composition 
ensemble. The parameters are n = 45, <5q = 0.01, S\ = 0.05, &2 = 0.25 and Q = (i, s> i). 



where R = ^ log M. The normal approximation in ( |102| > yields the approximation 



"~~ [ i s (Q)-R J 



(158) 



While the approximation rcu s in ( |145| l does not permit an explicit inversion, we can use the asymptotic expression 
in ( |149| l to obtain an approximate expression for n at rates above the critical rate. Setting the quantity 



(1 - p)p^J2irnc2 



to a target value e, we obtain 



ne 



2nE , r ia (Q,R.) 



1 

27rc 2 ((l-p)p)V 



Multiplying both sides by 2£y d and solving for n, we obtain the approximation 

1 2E [ r id {Q,R) \ 



-W 



2E^{Q,R) \ 2 nc 2 ((l - p)p) 



-\ 2 



(159) 



(160) 



(161) 



where W(z) is the Lambert function, defined to be the value w such that we w — z. As z — > 00, we have 
W(z) R3 log z, and hence for small values of e we can further approximate ( |161| l as 



loge 



log 



2£ I l id (Q,i?) 



(162) 



E^{Q,R) 2E**(Q,R) " 2 7rc 2 ((l - p)p)' 
This can be seen as a refinement of ( |157[ >. 

Equations ( |161| l-( fT62| > should be used with care. From Theorem |9] we expect the resulting approximations to be 
accurate for any R € (i? C r, Is{Q)) when the block length is large. However, \\59\ diverges as p —> or p — > 1, so 
for rates near R CI or I S {Q) one should instead use the approximation in ( |145[ ), particularly when the block length 
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is small. The divergence as p — > 1 is seen in Figure [5] where the curve corresponding to ( |159[ ) diverges from that 
of fcu s when the corresponding rate is close to i? cr = 0.192. However, as the block length and rate increase, the 
two curves quickly become indistinguishable. 

VII. Conclusion and Discussion 

Finite-length bounds on the random-coding error probability for mismatched decoders have been given. These 
bounds have been used to study the i.i.d. ensemble, constant-composition ensemble and cost-constrained ensembles. 
Error exponents and second-order coding rates and have been given for each ensemble, and connections have been 
drawn between each. While the best performance in terms of rates and exponents is generally achieved by the 
constant-composition ensemble, we have shown that one can recover these gains using cost-constrained coding 
with at most two cost functions. Thus, cost-constrained coding is an attractive alternative to constant-composition 
coding; the two ensembles have similar performance, but only the latter can be directly applied in the case of 
infinite (possibly continuous) alphabets. Finally, saddlepoint approximations to the random-coding bounds have 
been presented. These have shown to provide accurate approximations to the random-coding bounds with low 
computational complexity. 

Since the results in this paper are presented for a given input distribution Q, the extension to cost-constrained 
channels is straightforward. More concretely, suppose that each codeword x is constrained to satisfy the constraint 
i 2iLi c i x i) < r f° r some (system) cost function c(-). The i.i.d. ensemble is no longer suitable, since in all 
non-trivial cases it has a positive probability of producing codewords which violate the constraint. On the other 
hand, the results for the constant-composition ensemble remain unchanged provided that Q itself satisfies the cost 
constraint, i.e. J2 X Q( x ) c ( x ) < T. 

For the cost-constrained ensemble, the extension is less trivial but still straightforward. The main change required 
is a modification of the definition of T> n in ( fTo*) to include a constraint on the quantity ~ Y27=i c ( x i)- Unlike the 
pseudo-costs in ([T5J, where the sample mean can be higher or lower than the true mean, the system cost of each 
codeword is constrained to be less than or equal to its mean. That is, the additional constraint is given by 

1 n 

i=l x 

or similarly with both upper and lower bounds (e.g. — < — ^" =1 c{xi) — 4> c < for some S c ). Using this 
modified definition of T> n , one can prove the subexponential behavior of \i n in Proposition [T] and the exponents 
and second-order rates for the cost-constrained ensemble remain valid under any Q such that <f> c < T. In particular, 
one can recover the results of Theorems [7] and [8] using two pseudo-cost constraints and one system cost constraint 
in the definition of T> n . 
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Appendix 

A. Proof of Theorem [3] 

We begin by analyzing the cost-constrained ensemble. The codeword distribution in ( p"5j ) can be written as 

1 ™ 

Px(x) = —Y[Q(x t )l{P x eg n }, (164) 



where P x is the empirical distribution (type) of x, and 

n ' 

We define the sets 



Gn = {Px eV n (X) : |E F [ a/ (X)]-^|< ^,i = l,... (165) 



5 n (0») = {Ptr € P„(* xj;):P x e g„} (166) 

T n (PxY,g n ) = {?xy eV n (X x y) : P x G , Py = Py , [log K )] > E P [log q(X, Y)} } . (167) 

Roughly speaking, S n is the set of possible types of {X^ m \Y) for the ensemble, and T n is the set of types of 
{X {m '\Y) which lead to errors given that (X {m) ,Y) e T n {P XY ), where m is the transmitted message and m! 
is a different message. 

We have from ( |25| l and ( |28] l that p e = rcu(n, M). Expanding rcu(n, M) in terms of types, we obtain 



p e (n,M)= V[(X,Y) eT n (P XY )]^(P XY ), (168) 

where 

^j(P xy )= mm U(M-l) Yl P[(X,y)eT n (P XY )]\ (169) 

P X Y&T n {P xv ,Q n ) 

and y is an arbitrary sequence with type P Y . From ( |164[ ), the distribution of X is the same as that of X' ~ 
nr=i Q( x i) conditioned on the event that X 1 G Q n . From Proposition [I] the normalizing constant in ( |164[ ) satisfies 
/i„ = 1, and thus 



ip(Px Y ) = min 1, (M - 1) ^ P[(X', y) G T"(P X y)] (170) 

lin {l, (M - 1) exp ( - nD(P XY \\Q x jy)}, (171) 



max mm • 



where ( |171| i follows from the property of types in |15 Eq. (18)], and the fact that the number of joint types is 
polynomial in n. Applying a nearly identical argument to the probability in ( |168| l, we obtain 

p e (n,M)= max exp ( - nD{P XY \\Q x W))ip{P XY ). (172) 

PxY&S n (G n ) 

From the above-mentioned properties of types, the implied subexponential factor in ( |171[ ) can be bounded in both 
directions by polynomials which do not depend on P XY . We can therefore substitute ( |171| i into ( |172| i, yielding 

P e (n,e nR ) =exp(-nE r , n (Q,R,g n )), (173) 
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where 

a r ~ . i+ 

(174) 



E r<n (Q,R,Q n )= min _ min D{P XY \\Q x W) + D(P XY \\Q x P Y ) - R 

Px Y es„(g n ) p XY £T n (PxY,Q n ) 



To conclude the proof of ( |50l >, we need to show that the constraints |Ep[a;(X)] — 4>i\ < r in (T53J can be replaced 
by Ep[a;(X)] = 0;, regardless of the value of 5. This follows using the same argument as that used to replace 
minimizations over types by minimizations over all distributions fT3) ; a simple continuity argument shows that the 
exponential behavior is unaffected. 

The i.i.d. exponent in ( |4"8j ) follows from ( |50] > by setting L = 0. The constant-composition exponent in ( |49| ) 
follows by treating the constant-composition ensemble as a special case of the cost-constrained ensemble, using 
the parameters given in Section |n] Due to the constraints Px = Q and Px = Px in p3) and (46| i, the quantity 
I>(-Pxy HQ x Py) in ® can be replaced by I p (X; Y). 



B. Proof of Theorem [4] 

Using [a] + = max p6 [ 0jl ] pa, the convexity properties of the optimization problem, and Fan's minimax theorem 
(35), the expression in (49) can be written as 

E c r c {Q,R) = max E c c {Q,p) - pR (175) 

pS[0,l] 

K(Q,p)= min _ min £>(Pxy||Q X W) + pIs(X; Y). (176) 



where 



It remains to show that Eq c (Q,p) — Eq c (Q,p). We will show this by considering the minimizations in ( |176| > one 
at a time, and using Lagrange duality. In fact, we can immediately recognize 

min Ip(X\Y). (177) 

PxYeT(PxY) 

as the LM rate for the channel Py\x with input distribution Px', see (5). Thus, replacing this minimization with 
the equivalent dual expression in (4), we conclude that the minimization in ( |176| l is equivalent to 

a(x) \ 



mm sup > Pxy(x,v)\ log — — — — — — -— - p log — - — r ^ — 

«cv65«*W),>o,«(.)t^ V Q0*Wl/k) ^ g(a;,i/)-e«W 



(178) 

Using Fan's minimax theorem (35), we can swap the order of the minimum and the supremum. Hence, we first 
minimize the objective in ( |178| l over Pxy € S(Q) with s and a(x) fixed. Introducing a Lagrange multiplier rj(x) 
corresponding to the constraint Px = Q, the Lagrangian is given by 

x,y \ / x 

Setting Ap — T = gives 

b oP X Y(x,y) b 



PxY(x,y) E^Q(xMx,y) s e a ^ , . . „_„ 
1+l0g QWW ~ P S g (x,y)'e"M V(X) = °- (180) 
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Solving ( |180| l for P X y( x ,v) gives 

P XY ( X ,y) = Q( X )W(y\ X )e-^ ( ^^f^J ■ ^ 
Applying the constraint Px = Q to (JT8TJ, we obtain 

Y,W(y\ X )(^P^^Y = e^) (182) 



which implies 



Substituting ( |180| l into ( |179| l gives the simplified Lagrangian expression Lq = — 1 + Q(x)r](x), and applying 
dl 83b, we obtain 



^-E»)^*)(»r). 

Since the optimization under consideration is a convex optimization problem, the duality gap is zero [11]. The 
proof is concluded by taking the supremum over (s,a). 

C. Proof of Theorem [5] 

It suffices to prove the theorem for Q with the same support as Q, since otherwise the divergence in ( |82| ) is 
infinite and Q cannot achieve the maximum. We fix Q{ X ) such that supp(<3) = supp(Q), and define a(x) such 
that 

e «wOM = e 5W. (1 85 ) 

Q(x) 

Substituting ( |185| l into ((55]) yields 



E?(Q,p)= sup -YQ( X )logY 8^ W^a) ( E>Q(«)g(«,y)'e 



(186) 



su P -X)q(»)i°gE ( ^Vfsi^ Y - (i + pMQWQ) d87) 

s >0,5(-) V y Q(x) \ q{x,yYe a{ - x ) J 



> sup - log Q(x)W(y\x) I E ^ Q f )q tllf il£) ) ~a + P)D(Q\\Q), (188) 
s>o,a(-) jrj \ q{ X ,y) s e a W J 

where ( |186| l follows since, for any given (Q,Q) pair, maximizing over a(-) is equivalent to maximizing over a( ), 
and ( |188| l follows from Jensen's inequality. It remains to show that Q(x) ( |188| l holds with equality when Q is 
chosen to maximize ( |187) . It suffices to show that the same holds true when the supremum over s on each side 
is replaced by any given value of s. We will show that equality holds in ( |188| l when Q and a( ) are chosen to 
maximize the objective in ( [188) , and then show that the resulting a(-) (which is computed from ( |185| l) maximizes 
the objective in ( |55] l (with respect to ai(-)) for the given value of s. We define 

f{ x ,Q,a) = Y, w{ -y\ x )^y)~ ps fe^M^r^ • ( 189 > 
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The optimization problems corresponding to ( |55] l and ( |188| > can respectively be written as 

■ 5^ Q(a:) ( -po(a:)+ log /(a:, Q, a)) (190) 



maximize 

o(-) 

maximize- logVQ(a;)e^ w /(a;,Q, a) - (1 + p)£)(Q||Q). (191) 

Q,3(0 x 

To obtain the KKT conditions, the following derivatives will be useful. 

,)i[r - Q -" ] ^J2w(y\x)q(x : y)-P s p(j2Q(^,yye Ui A q(a/,y)'e^ = g(x,x',Q,a) (192) 



5Q(x>) 

5f(x,Q,a) 



Q{x')g{x>x',Q,a) (193) 



Sa(x') 

The Lagrangians corresponding to ( |190| > and ( |191| l are respectively given by 

ii = - Q( x ) ( - P a ^ + lo S <2, «)) (194) 
L 2 = - log ^ Q^e"'^/^, Q, a) - (1 + p) £ Q(x) log Sj^ + M ( £ Q(x) - l) , (195) 

where p is a Lagrange multiplier. 

We begin by analyzing the KKT conditions of ( |191) , While the objective in non-concave in (Q,a), the KKT 
conditions are still necessary for optimality. However, we must rule out the possibility that the maximization is 
obtained by a limiting value, e.g. as \a(x)\ grows unbounded for some x. We first observe that each Q(x) must 
remain bounded away from zero for x G supp(Q). To see this, we note that (i) the first term in ( |191| l can easily 
be upper bounded by a finite constant, and (ii) as Q(x) — > for x £ supp(Q), the divergence term in ( |191[ ) tends 
toward— oo. Secondly, since the objective is unchanged if the same constant value is added to each a(x) (cf. ( |188| >), 
it suffices to fix the highest value of a(x) to zero and show that the maximum cannot be achieved in the limit as the 
remaining values approach — oo. Indeed, since our setup assumes q(x, y) > wherever ^(yjx) > (cf. Section 



I-Ai, a(x) —> — oo would imply a division by a vanishing quantity in ( |188[ ), yielding an objective of — oo. 

e-^f(x',Q,a) + E x Q(x)e-^g(x,x\Q,a) | | Q(s') Q 
E x Q(x)e- p ~ a{x) f(x,Q,a) Q(x') 

Settin g = 8 ives 

-1 

and hence _ _ 

J2 x Q(x)e-P~ a ^g(x 1 x / ,Q 1 a) 



Q(x , )pe-P^f(x\Q,a) + YQ(x)e-P^Q(x')g(x,x\Q,a)) = 0, (197) 



= P- (198) 

P< x ')f( x ',Q,a) 



Substituting ( |198| l into ( |196| l gives 



-pa(x') f / / Q ~) Q(x') 
(1 + p) = J 4, ~ +(l + p )^- L +p = 0. (199) 
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Multiplying both sides by Q(x') and summing over x' gives /i = 0, and hence 



for all x' . It follows from \2QQ\ that the quantity 



(200) 



9^1 e ~^f(x',Q,a) (201) 

is independent of x' , which is a sufficient condition for Jensen's inequality to hold with equality in ( |188| l. 

We now turn to the KKT conditions for ( |55j ). We will show that if (Q,a) satisfies the KKT conditions for ( jl91[ >, 
then the cost function a( ) which is computed using ( |185[ ) satisfies the KKT conditions for ( |190| l. We denote such 
a (Q,a) pair by (Q*,a*), and we denote the corresponding a(-) by a*(-). 

Settin g 4% = § ives 

— /m^5 = (202) 



or equivalently 



^ Q(x)g(x,x',Q,a) 

? /(*,Q,a) (2 ° 3) 



We must show that a*(-) satisfies ( |203| > for all a;'; this will complete the proof since ( |190| > is a convex optimization 
problem with affine constraints, and hence the KKT conditions are both necessary and sufficient (TTJ. Using ( |185[ ) 
and the definitions of / and g, we have 

f(x,Q,a*)=f(x,Q*,a*) (204) 

g(x,x',Q,a*) = ^^g{x,x\ Q\a*). (205) 

Thus, a*{-) satisfies ( |203| > if and only if (Q*,a*) satisfies 

Q*(x') v-^ Q(x) 3 (x,x',g*,a*) 



— V l^illll = (206) 



Q(*0 V f(x,Q*,a*) 
or equivalently 

g^Q y Q*(x)e-P^^g(x,x',Q* 1 a*) = 
Q{X>) * ^-e-P^)f(x,Q*,a*) 
Since the quantity in ( |201| > is independent of a;' under (Q*,a*), it follows that ( |207| i holds if and only if 



0(a; ' ) * %^e-pW)f(x>,Q*,a*) 

P- (209) 



or equivalently 

E x Q*(x)e-" v ^g(x,x',Q*,a*) 



e-P a *( x ')f(x',Q*,a*) 

Finally, since ( |209| l coincides with ( jl98| >, which in turn was a result of the KKT conditions for ( |191[ ), it follows 
that ( |209| > holds, and hence a*(-) satisfies ( |203| > for all a;'. Thus, a*(-) maximizes the objective in ( |190| >. 
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D. Proof of Theorem [9] 

The proof of the second part of the theorem is based based on the analysis of Bahadur and Rao f36| , while the 
proof of the first part is more straightforward. In both cases, it is easy to show that the use of M in place of M — 1 
in the definition of £ n does not affect the statement of the theorem. Thus, throughout the proof we focus on the 
bound rcu s (n, M + 1). 

1) Below the Critical Rate: Using Gallager's properties of error exponents |T[ Sec. 5.6], we have p = 1, c\ < 
and C2 > for all R < i? cr , and an asymptotic expansion of a lld (p, s) as n — > oo yields ( |148| l. To prove ( |147| i, 
we need to show that 



rcu s (n,M + l) = (l + o(l)) cxp (n(E$ d (Q,l,s) 



(210) 



The upper bound follows immediately by applying min{l, a} < a to |26|. To obtain a matching lower bound, we 
write 



rcu s (n,M+ 1) =P %(X,Y) <logM 
= W>\%(X,Y) < logM 



> 



%(X,Y) <logM 



ME 
ME 
ME 



t{i n a {X,Y) >logM} 



ME 



<( X ' Y h{t^{X,Y) <logM} 



- M 



E 



(l+p')i™{X,Y) 



(211) 
(212) 
(213) 



where ( |211| i follows by splitting the min{l, ■} in (|26j, ( |212| i follows since 1{A} + 1{A C } = 1, and ( |213| l follows 
for any p' > by upper bounding the indicator function. 

The second term in the summation in ( |213[ ) is simply the right-hand side of ( |210| i with the o(l) term omitted, so 
it only remains to show that the other two terms decay at a faster rate. Using the strong large deviations result of 



1 36 , the first term decays with an exponent at least as high as EQ d (Q, 1, s) — R, with an additional -j)= pre-factor. 
Furthermore, since the sphere-packing bound exceeds the random-coding bound below the critical rate (T| Sec. 5], 
there exists a p' which makes the third term in ( |213| l decay with an exponent exceeding EQ d (Q, 1, s) — R. 

2) Above the Critical Rate: Using Gallager's properties of error exponents Jl] Sec. 5.6], we have p = 1, c\ = 
and C2 > for all R > R CT , and an asymptotic expansion of a lld (p, s) as n — > oo yields ( [149) 1. To prove ( |147| >, 



we combine the techniques of [36] with the refined version of the central limit theorem given in |T6] Sec. XVI.4, 
Thm. 1]. 

Using the expression for rcu s in ( |130| >, we have 



rcu s (n,M + 1) = 



iR-J2i s (Xi,Yi)>hgU 



(214) 



This expression resembles the upper tail probability of an i.i.d. sum of random variables, for which asymptotic 
estimates were given by Bahadur and Rao |36|. Due to the presence of the logU term in ( |214[ ), the results of |36| 
are not directly applicable. However, by performing a similar analysis, we will obtain the desired result. 

Let F(t) denote the cumulative distribution function (CDF) of R — i s (X, Y), and let Z\, ■ ■ ■ , Z n be distributed 
according to the tilted CDF 



Fz{z) 



(215) 
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where we write £y ld as a shorthand for cxp (i?Q ld (<5, p, s) — /Si?). Using an identical argument to [36 
we can write 



where 



In. = E 



rcu s (rc,M + 1) = I n e 

>logl7 



Using an integration by parts, we obtain (36 



J„ = jT pay/nj^ e" pW ^ (F n (z) - F n ) dzdtt, 



Lemma 2], 
(216) 

(217) 
(218) 



where a 2 = Var[Z], and F n (z) is the CDF of 2=^5. . 

Let Q(z) denote the CDF of a zero-mean unit-variance Gaussian random variable. From 1 16 Sec. XVI.4, Thm. 
1], we have 

F n (z) = $(z) + G n (z) + F n (z), (219) 



where F n {z) — o(n 2 ) uniformly in z, and 



G n {z) 



A 



K 



1 - z 2 )e 



(220) 



for some constant K depending only on the second and third moments of Z, both of which are finite for any DMC. 
Substituting \2\9\ into ( |218] >, we obtain 

1 



In = h,n + h,n + O 



where 



h. 



A 



A 



dzdu, 



p<J\fn 



GJz)-G 



\<j\ nl I 



dzdu. 



(221) 



(222) 



(223) 



To evaluate ([222} we use integration by parts to obtain an expression of the same form as ( |217| >, with each Zj 
replaced by a zero-mean random variable with variance a 2 . A direct evaluation of the resulting expectation yields 
precisely the pre-factor in ( |146| l, with and a 2 in place of c\ and C2 respectively. Evaluating ( |141| i and ( |142[ ) 



directly, and using the expression for a 2 described by |36 Eqs. (4), (8)], it is easily verified that these coefficients 
coincide. From ( |149[ ) and ( |221| ), it only remains to show that 7 2 ,n = 



Following [36 Eq. (28)], we can use integration by parts to write 

"1 poo 



l2,n 



1 

2^ 







e - pa ^ nz G' n (z)dzdu, 



where G' n (z) is the derivative of G n (z), namely 



G' n {z) = ^-Zz)e 



(224) 



(225) 
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Changing the order of integration in ( |224[ ), we obtain 

roc / .min{l,e" v/ " i } 



J 2 =_—/ I " ' e-P a ^ z (z 3 -3z)e-^ z ' 2 dudz (226) 



oo ^0 

oo 



2 



(227) 



tc r — 

— / e -P a ^ z (z 3 -3z)min{l,e a ^ z }e-i z2 dz 
nV n J-oo 

= ^—{ [ e-^^-fz 3 -3z)e-^ z2 dz+ [ e^^^U 3 -3z)e-i z2 dz) (228) 

Evaluating the integrals in ( |228| l, we conclude that both are OinT 1 ), and hence I^.n — 0(n~i). Thus, using ( |221| i, 
we conclude that /„ = Ii n + o(n~i). Since Ii n is the desired pre-factor, and decays as (cf. ( |149| l), we 
conclude that lim^^cx, j^- = 1, thus concluding the proof. 
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